just adults

Now that the meta data is organized, I can pretty clearly see that some of the issues are caused by juvenile samples.

Let’s re-run the analysis with them removed, as a test case.

Load libraries and data

library(tidyverse)
library(rubias)
# meta data
meta <- readRDS("../new_baseline_data/processed/meta-data-tibble.rds")
# genotype data in 2-column format for rubias
new2col <- read_csv("csv_outputs/new2col.csv")

Parsed with column specification:
cols(
  .default = col_integer(),
  sample_type = col_character(),
  repunit = col_character(),
  collection = col_character(),
  indiv = col_character()
)
See spec(...) for full column specifications.

How many of those are juveniles?

There are 160 juvenile samples

If I remove samples that are juveniles, what are my numbers per species?

new2col %>%
  semi_join(., meta, by = c("indiv" = "NMFS_DNA_ID")) %>%
  #anti_join(., juvies, by = "indiv") %>%
  group_by(collection) %>%
  tally() %>%
  rename(total_samples = n) %>%
  left_join(., juv_samples, by = "collection") %>%
  mutate(remaining_samples = total_samples-juv_samples) %>%
  arrange(remaining_samples)

We lose reedi and wilsoni when we exclude juveniles, and drop down to just 2 samples of crameri and five of serriceps.

However, generally including juveniles is a bad idea. So here’s what I’ll do: For any species for which we end up with fewer than 5 samples, I will include juveniles up to that number.

Now that I have those samples selected, I can remove all juvenile samples and put back those from the juvs_to_keep

Keeping just the juveniles from those three species, we have a total of 1,534 samples from 54 species.

Let’s try out the self-assignment with that dataset:

minus_juvs_selfass <- self_assign(dataset, gen_start_col = 5)

Summary Statistics:

1534 Individuals in Sample

90 Loci: Plate_1_A01_Sat_GW603857_consensus.1, Plate_1_A11_Sat_GE820299_consensus.1, Plate_2_A09_Sat_EW986980_consensus.1, Plate_2_C08_Sat_EW987116_consensus.1, Plate_2_G06_Sat_EW987118_consensus.1, Plate_3_C03_Sat_GE798118_consensus.1, Plate_4_E10_Sat_EW976030_consensus.1, Plate_4_G06_Sat_EW976181_consensus.1, tag_id_1049.1, tag_id_108.1, tag_id_1184.1, tag_id_1229.1, tag_id_1272.1, tag_id_1366.1, tag_id_1428.1, tag_id_143.1, tag_id_1441.1, tag_id_1449.1, tag_id_1471.1, tag_id_1498.1, tag_id_1558.1, tag_id_1576.1, tag_id_1598.1, tag_id_1604.1, tag_id_1613.1, tag_id_162.1, tag_id_1652.1, tag_id_170.1, tag_id_1708.1, tag_id_1748.1, tag_id_1751.1, tag_id_1762.1, tag_id_179.1, tag_id_1804.1, tag_id_1808.1, tag_id_1810.1, tag_id_1836.1, tag_id_1850.1, tag_id_1880.1, tag_id_1889.1, tag_id_1915.1, tag_id_1950.1, tag_id_1961.1, tag_id_1966.1, tag_id_1982.1, tag_id_1994.1, tag_id_1999.1, tag_id_2008.1, tag_id_2009.1, tag_id_2017.1, tag_id_2062.1, tag_id_2082.1, tag_id_2114.1, tag_id_2134.1, tag_id_2155.1, tag_id_2178.1, tag_id_2182.1, tag_id_220.1, tag_id_2203.1, tag_id_221.1, tag_id_2214.1, tag_id_2237.1, tag_id_2247.1, tag_id_2258.1, tag_id_2301.1, tag_id_2319.1, tag_id_2368.1, tag_id_2499.1, tag_id_250.1, tag_id_2607.1, tag_id_2635.1, tag_id_265.1, tag_id_325.1, tag_id_402.1, tag_id_410.1, tag_id_436.1, tag_id_55.1, tag_id_572.1, tag_id_67.1, tag_id_770.1, tag_id_788.1, tag_id_843.1, tag_id_855.1, tag_id_874.1, tag_id_875.1, tag_id_879.1, tag_id_913.1, tag_id_942.1, tag_id_981.1, tag_id_987.1

54 Reporting Units: aleutianus, alutus, atrovirens, auriculatus, aurora, babcocki, borealis, carnatus, caurinus, chlorostictus, chrysomelas, constellatus, crameri, dallii, diaconus, diploproa, elongatus, emphaeus, ensifer, entomelas, flavidus, goodei, hopkinsi, jordani, levis, maliger, melanops, melanostictus, melanostomus, miniatus, moseri, mystinus, nebulosus, nigrocinctus, oculatus, ovalis, paucispinis, pinniger, polyspinis, proriger, rastrelliger, rosaceus, ruberrimus, rubrivinctus, rufinanus, rufus, saxicola, semicinctus, serranoides, serriceps, umbrosus, zacentrus, reedi, wilsoni

54 Collections: aleutianus, alutus, atrovirens, auriculatus, aurora, babcocki, borealis, carnatus, caurinus, chlorostictus, chrysomelas, constellatus, crameri, dallii, diaconus, diploproa, elongatus, emphaeus, ensifer, entomelas, flavidus, goodei, hopkinsi, jordani, levis, maliger, melanops, melanostictus, melanostomus, miniatus, moseri, mystinus, nebulosus, nigrocinctus, oculatus, ovalis, paucispinis, pinniger, polyspinis, proriger, rastrelliger, rosaceus, ruberrimus, rubrivinctus, rufinanus, rufus, saxicola, semicinctus, serranoides, serriceps, umbrosus, zacentrus, reedi, wilsoni

6.27770534550196% of allelic data identified as missing

What I’m really curious about is whether that cleaned up some of the z-score issues?

# which samples still have z-score issues 
z_bummers <- minus_juvs_selfass %>%
  ungroup() %>%
  filter(scaled_likelihood > 0.95) %>%
  filter(z_score < -3) %>%
  select(indiv, collection, inferred_collection, z_score, scaled_likelihood, n_miss_loci) %>%
  arrange(z_score) 
z_bummers

13 samples with z-scores < -3.

If the average amount of missing data is ~6% (5.5 loci), then the samples with 9, 11, and 13 missing loci are potentially significant, but missing data doesn’t explain the hopkinsi samples.

Although I poked around to figure out what was causing the low z-scores, utimately, I should just remove them. (unless that is too circular, but for the reference baseline, we’ll go ahead because it might be some indication that the samples are not what they should be based on meta data.)

d2 <- dataset %>%
  ungroup() %>%
  anti_join(., z_bummers, by = "indiv")
d2 

Removing samples with z-scores < -3 leaves 1,521 samples.

Try self-assignment again without those

assign2 <- self_assign(d2, gen_start_col = 5)

Summary Statistics:

1521 Individuals in Sample

90 Loci: Plate_1_A01_Sat_GW603857_consensus.1, Plate_1_A11_Sat_GE820299_consensus.1, Plate_2_A09_Sat_EW986980_consensus.1, Plate_2_C08_Sat_EW987116_consensus.1, Plate_2_G06_Sat_EW987118_consensus.1, Plate_3_C03_Sat_GE798118_consensus.1, Plate_4_E10_Sat_EW976030_consensus.1, Plate_4_G06_Sat_EW976181_consensus.1, tag_id_1049.1, tag_id_108.1, tag_id_1184.1, tag_id_1229.1, tag_id_1272.1, tag_id_1366.1, tag_id_1428.1, tag_id_143.1, tag_id_1441.1, tag_id_1449.1, tag_id_1471.1, tag_id_1498.1, tag_id_1558.1, tag_id_1576.1, tag_id_1598.1, tag_id_1604.1, tag_id_1613.1, tag_id_162.1, tag_id_1652.1, tag_id_170.1, tag_id_1708.1, tag_id_1748.1, tag_id_1751.1, tag_id_1762.1, tag_id_179.1, tag_id_1804.1, tag_id_1808.1, tag_id_1810.1, tag_id_1836.1, tag_id_1850.1, tag_id_1880.1, tag_id_1889.1, tag_id_1915.1, tag_id_1950.1, tag_id_1961.1, tag_id_1966.1, tag_id_1982.1, tag_id_1994.1, tag_id_1999.1, tag_id_2008.1, tag_id_2009.1, tag_id_2017.1, tag_id_2062.1, tag_id_2082.1, tag_id_2114.1, tag_id_2134.1, tag_id_2155.1, tag_id_2178.1, tag_id_2182.1, tag_id_220.1, tag_id_2203.1, tag_id_221.1, tag_id_2214.1, tag_id_2237.1, tag_id_2247.1, tag_id_2258.1, tag_id_2301.1, tag_id_2319.1, tag_id_2368.1, tag_id_2499.1, tag_id_250.1, tag_id_2607.1, tag_id_2635.1, tag_id_265.1, tag_id_325.1, tag_id_402.1, tag_id_410.1, tag_id_436.1, tag_id_55.1, tag_id_572.1, tag_id_67.1, tag_id_770.1, tag_id_788.1, tag_id_843.1, tag_id_855.1, tag_id_874.1, tag_id_875.1, tag_id_879.1, tag_id_913.1, tag_id_942.1, tag_id_981.1, tag_id_987.1

54 Reporting Units: aleutianus, alutus, atrovirens, auriculatus, aurora, babcocki, borealis, carnatus, caurinus, chlorostictus, chrysomelas, constellatus, crameri, dallii, diaconus, diploproa, elongatus, emphaeus, ensifer, entomelas, flavidus, goodei, hopkinsi, jordani, levis, maliger, melanops, melanostictus, melanostomus, miniatus, moseri, mystinus, nebulosus, nigrocinctus, oculatus, ovalis, paucispinis, pinniger, polyspinis, proriger, rastrelliger, rosaceus, ruberrimus, rubrivinctus, rufinanus, rufus, saxicola, semicinctus, serranoides, serriceps, umbrosus, zacentrus, reedi, wilsoni

54 Collections: aleutianus, alutus, atrovirens, auriculatus, aurora, babcocki, borealis, carnatus, caurinus, chlorostictus, chrysomelas, constellatus, crameri, dallii, diaconus, diploproa, elongatus, emphaeus, ensifer, entomelas, flavidus, goodei, hopkinsi, jordani, levis, maliger, melanops, melanostictus, melanostomus, miniatus, moseri, mystinus, nebulosus, nigrocinctus, oculatus, ovalis, paucispinis, pinniger, polyspinis, proriger, rastrelliger, rosaceus, ruberrimus, rubrivinctus, rufinanus, rufus, saxicola, semicinctus, serranoides, serriceps, umbrosus, zacentrus, reedi, wilsoni

6.28606910658193% of allelic data identified as missing

Warning message:
In `[<-.data.frame`(`*tmp*`, is_list, value = list(`11` = "<>")) :
  replacement element 1 has 1 row to replace 0 rows

What’s the proportion correctly assigned now?

assign2 %>%
  ungroup() %>%
  filter(scaled_likelihood > 0.95) %>%
  filter(collection == inferred_collection)

1457/1521

[1] 0.9579224

98% assigned correctly at the 50% level 96% assigned correctly at the 99% level

Are all of the misassignments gopher/black-and-yellow?

assign2 %>%
  filter(collection != inferred_collection) %>%
  filter(scaled_likelihood > 0.5)

Yes. All of the misassignments are gopher/black-and-yellow.

Make a single repunit for the gopher/black-and-yellow samples

# change the carnatus reporting unit to chrysomelas
gby_repu <- d2 %>%
  mutate(repunit = ifelse(repunit == "carnatus", "gby", repunit)) %>%
  mutate(repunit = ifelse(repunit == "chrysomelas", "gby", repunit)) %>%
  mutate(collection = ifelse(collection == "carnatus", "gby", repunit)) %>%
  mutate(collection = ifelse(collection == "chrysomelas", "gby", repunit))
# confirm that the repunit is changed but the collection is not.
gby_repu %>%
  filter(collection == "carnatus")

Try self-assignment with that

assign3 <- self_assign(gby_repu, gen_start_col = 5)

Summary Statistics:

1521 Individuals in Sample

90 Loci: Plate_1_A01_Sat_GW603857_consensus.1, Plate_1_A11_Sat_GE820299_consensus.1, Plate_2_A09_Sat_EW986980_consensus.1, Plate_2_C08_Sat_EW987116_consensus.1, Plate_2_G06_Sat_EW987118_consensus.1, Plate_3_C03_Sat_GE798118_consensus.1, Plate_4_E10_Sat_EW976030_consensus.1, Plate_4_G06_Sat_EW976181_consensus.1, tag_id_1049.1, tag_id_108.1, tag_id_1184.1, tag_id_1229.1, tag_id_1272.1, tag_id_1366.1, tag_id_1428.1, tag_id_143.1, tag_id_1441.1, tag_id_1449.1, tag_id_1471.1, tag_id_1498.1, tag_id_1558.1, tag_id_1576.1, tag_id_1598.1, tag_id_1604.1, tag_id_1613.1, tag_id_162.1, tag_id_1652.1, tag_id_170.1, tag_id_1708.1, tag_id_1748.1, tag_id_1751.1, tag_id_1762.1, tag_id_179.1, tag_id_1804.1, tag_id_1808.1, tag_id_1810.1, tag_id_1836.1, tag_id_1850.1, tag_id_1880.1, tag_id_1889.1, tag_id_1915.1, tag_id_1950.1, tag_id_1961.1, tag_id_1966.1, tag_id_1982.1, tag_id_1994.1, tag_id_1999.1, tag_id_2008.1, tag_id_2009.1, tag_id_2017.1, tag_id_2062.1, tag_id_2082.1, tag_id_2114.1, tag_id_2134.1, tag_id_2155.1, tag_id_2178.1, tag_id_2182.1, tag_id_220.1, tag_id_2203.1, tag_id_221.1, tag_id_2214.1, tag_id_2237.1, tag_id_2247.1, tag_id_2258.1, tag_id_2301.1, tag_id_2319.1, tag_id_2368.1, tag_id_2499.1, tag_id_250.1, tag_id_2607.1, tag_id_2635.1, tag_id_265.1, tag_id_325.1, tag_id_402.1, tag_id_410.1, tag_id_436.1, tag_id_55.1, tag_id_572.1, tag_id_67.1, tag_id_770.1, tag_id_788.1, tag_id_843.1, tag_id_855.1, tag_id_874.1, tag_id_875.1, tag_id_879.1, tag_id_913.1, tag_id_942.1, tag_id_981.1, tag_id_987.1

53 Reporting Units: aleutianus, alutus, atrovirens, auriculatus, aurora, babcocki, borealis, gby, caurinus, chlorostictus, constellatus, crameri, dallii, diaconus, diploproa, elongatus, emphaeus, ensifer, entomelas, flavidus, goodei, hopkinsi, jordani, levis, maliger, melanops, melanostictus, melanostomus, miniatus, moseri, mystinus, nebulosus, nigrocinctus, oculatus, ovalis, paucispinis, pinniger, polyspinis, proriger, rastrelliger, rosaceus, ruberrimus, rubrivinctus, rufinanus, rufus, saxicola, semicinctus, serranoides, serriceps, umbrosus, zacentrus, reedi, wilsoni

53 Collections: aleutianus, alutus, atrovirens, auriculatus, aurora, babcocki, borealis, gby, caurinus, chlorostictus, constellatus, crameri, dallii, diaconus, diploproa, elongatus, emphaeus, ensifer, entomelas, flavidus, goodei, hopkinsi, jordani, levis, maliger, melanops, melanostictus, melanostomus, miniatus, moseri, mystinus, nebulosus, nigrocinctus, oculatus, ovalis, paucispinis, pinniger, polyspinis, proriger, rastrelliger, rosaceus, ruberrimus, rubrivinctus, rufinanus, rufus, saxicola, semicinctus, serranoides, serriceps, umbrosus, zacentrus, reedi, wilsoni

6.28606910658193% of allelic data identified as missing

Accurate assignment?

assign3 %>%
  filter(scaled_likelihood > 0.95) %>%
  filter(repunit == inferred_repunit)

100% at the 0.95 likelihood level

Well, either I switch the collection information, in which case, I have 100% assignment, or I move forward with the lower accuracy.

But I think it’s more appropriate to keep the collections intact? Maybe I will describe it both ways in the manuscript:

When carnatus and chrysomelas are individual collections with the ability to assign to a single reporting unit, 96.4% of assignments are accurate at a 95% likelihood threshold.

When carnatus and chrysomelas are considered a single collection (gby), we attain 100% accuracy in assignments at the 95% likelihood threshold.

Downsample

What if I take a maximum of 32 samples per species?

# which species have fewer than 32 samples?
sm_grps <- d2 %>%
  group_by(collection) %>%
  tally() %>%
  arrange(n) %>%
  filter(n < 33)
# make a 2-col dataframe with just those groups
sm_d2 <- d2 %>%
  semi_join(., sm_grps) %>%
  ungroup()

Joining, by = "collection"

# which species have more than 32 samples?
lrg_grps <- d2 %>%
  group_by(collection) %>%
  tally() %>%
  arrange(n) %>%
  filter(n > 32)
  
# downsample those groups
down_sam <- d2 %>%
  semi_join(., lrg_grps) %>%
  group_by(collection) %>%
  sample_n(., 32, replace = FALSE) %>%
  ungroup()

Joining, by = "collection"

# and add the data from the other groups back into the dataframe
down_data <- sm_d2 %>%
  bind_rows(down_sam) #%>%
  # group_by(collection) %>%
  # tally() %>%
  # arrange(desc(n))

Great. that downsampling worked. There are 1,005 samples from 54 species. Now do the self-assignment:

assign_down <- self_assign(down_data, gen_start_col = 5)

Summary Statistics:

1005 Individuals in Sample

90 Loci: Plate_1_A01_Sat_GW603857_consensus.1, Plate_1_A11_Sat_GE820299_consensus.1, Plate_2_A09_Sat_EW986980_consensus.1, Plate_2_C08_Sat_EW987116_consensus.1, Plate_2_G06_Sat_EW987118_consensus.1, Plate_3_C03_Sat_GE798118_consensus.1, Plate_4_E10_Sat_EW976030_consensus.1, Plate_4_G06_Sat_EW976181_consensus.1, tag_id_1049.1, tag_id_108.1, tag_id_1184.1, tag_id_1229.1, tag_id_1272.1, tag_id_1366.1, tag_id_1428.1, tag_id_143.1, tag_id_1441.1, tag_id_1449.1, tag_id_1471.1, tag_id_1498.1, tag_id_1558.1, tag_id_1576.1, tag_id_1598.1, tag_id_1604.1, tag_id_1613.1, tag_id_162.1, tag_id_1652.1, tag_id_170.1, tag_id_1708.1, tag_id_1748.1, tag_id_1751.1, tag_id_1762.1, tag_id_179.1, tag_id_1804.1, tag_id_1808.1, tag_id_1810.1, tag_id_1836.1, tag_id_1850.1, tag_id_1880.1, tag_id_1889.1, tag_id_1915.1, tag_id_1950.1, tag_id_1961.1, tag_id_1966.1, tag_id_1982.1, tag_id_1994.1, tag_id_1999.1, tag_id_2008.1, tag_id_2009.1, tag_id_2017.1, tag_id_2062.1, tag_id_2082.1, tag_id_2114.1, tag_id_2134.1, tag_id_2155.1, tag_id_2178.1, tag_id_2182.1, tag_id_220.1, tag_id_2203.1, tag_id_221.1, tag_id_2214.1, tag_id_2237.1, tag_id_2247.1, tag_id_2258.1, tag_id_2301.1, tag_id_2319.1, tag_id_2368.1, tag_id_2499.1, tag_id_250.1, tag_id_2607.1, tag_id_2635.1, tag_id_265.1, tag_id_325.1, tag_id_402.1, tag_id_410.1, tag_id_436.1, tag_id_55.1, tag_id_572.1, tag_id_67.1, tag_id_770.1, tag_id_788.1, tag_id_843.1, tag_id_855.1, tag_id_874.1, tag_id_875.1, tag_id_879.1, tag_id_913.1, tag_id_942.1, tag_id_981.1, tag_id_987.1

54 Reporting Units: aleutianus, alutus, auriculatus, aurora, babcocki, borealis, caurinus, chlorostictus, constellatus, crameri, dallii, diaconus, diploproa, elongatus, emphaeus, ensifer, entomelas, flavidus, hopkinsi, jordani, maliger, melanops, melanostictus, melanostomus, miniatus, moseri, mystinus, nebulosus, nigrocinctus, oculatus, paucispinis, pinniger, polyspinis, proriger, rastrelliger, rosaceus, ruberrimus, rubrivinctus, rufinanus, rufus, saxicola, semicinctus, serriceps, umbrosus, zacentrus, reedi, wilsoni, atrovirens, carnatus, chrysomelas, goodei, levis, ovalis, serranoides

54 Collections: aleutianus, alutus, auriculatus, aurora, babcocki, borealis, caurinus, chlorostictus, constellatus, crameri, dallii, diaconus, diploproa, elongatus, emphaeus, ensifer, entomelas, flavidus, hopkinsi, jordani, maliger, melanops, melanostictus, melanostomus, miniatus, moseri, mystinus, nebulosus, nigrocinctus, oculatus, paucispinis, pinniger, polyspinis, proriger, rastrelliger, rosaceus, ruberrimus, rubrivinctus, rufinanus, rufus, saxicola, semicinctus, serriceps, umbrosus, zacentrus, reedi, wilsoni, atrovirens, carnatus, chrysomelas, goodei, levis, ovalis, serranoides

8.54063018242123% of allelic data identified as missing

What’s our accuracy?

assign_down %>%
  filter(repunit == inferred_repunit) %>%
  filter(scaled_likelihood > 0.95)

979/1005

[1] 0.9741294

z-scores?

Damn. There are still three samples, all of which are the hopkinsi. I should remove them because (as seen in the PCA) they are clearly wonky somehow (done in 08-pca-w-downsampled-dataset.Rmd)

I think I should move forward with this downsampled dataset because a general reference baseline doesn’t need 200 samples of kelp rockfish and carrying around those genotypes is cumbersome.

Remove those last four samples with wonky z-scores:

down_no_z_data <- down_data %>%
  anti_join(., z_to_remove)

Joining, by = c("repunit", "collection", "indiv")

In this dataset there are 1002 total samples.

Output dataset for figures and other analyses:

dat3 <- down_no_z_data %>%
  ungroup() %>%
  select(-repunit, -sample_type) %>%
  mutate(sample = paste0(collection, indiv)) %>%
  select(183, 1:182)
  # need to reorder the collections to be in phylogenetic order

# set the levels explicitly
dat3$collection <- factor(dat3$collection, levels = c("atrovirens", "chrysomelas", "carnatus", "caurinus", "maliger", "nebulosus", "dallii","rastrelliger", "auriculatus", "saxicola", "semicinctus", "elongatus", "miniatus", "pinniger", "oculatus", "constellatus", "umbrosus", "rosaceus", "chlorostictus", "ensifer", "babcocki", "nigrocinctus", "rubrivinctus", "serriceps", "rufus", "ovalis", "hopkinsi", "rufinanus", "moseri", "jordani", "paucispinis", "goodei", "levis", "entomelas", "mystinus", "diaconus", "flavidus", "serranoides", "melanops", "ruberrimus", "aurora", "diploproa", "melanostomus", "reedi", "crameri", "polyspinis", "alutus", "melanostictus", "aleutianus", "emphaeus", "wilsoni", "zacentrus", "proriger", "borealis"))
ordered_frame <- with(dat3, dat3[order(collection),])
ordered_frame %>%
  select(-collection, -indiv) %>%
  write_csv("csv_outputs/ordered_downsampled_sppID_data.csv")

When we do the self-assignment on this dataset, we get:

assign4 <- self_assign(down_no_z_data, gen_start_col = 5)

Summary Statistics:

1002 Individuals in Sample

90 Loci: Plate_1_A01_Sat_GW603857_consensus.1, Plate_1_A11_Sat_GE820299_consensus.1, Plate_2_A09_Sat_EW986980_consensus.1, Plate_2_C08_Sat_EW987116_consensus.1, Plate_2_G06_Sat_EW987118_consensus.1, Plate_3_C03_Sat_GE798118_consensus.1, Plate_4_E10_Sat_EW976030_consensus.1, Plate_4_G06_Sat_EW976181_consensus.1, tag_id_1049.1, tag_id_108.1, tag_id_1184.1, tag_id_1229.1, tag_id_1272.1, tag_id_1366.1, tag_id_1428.1, tag_id_143.1, tag_id_1441.1, tag_id_1449.1, tag_id_1471.1, tag_id_1498.1, tag_id_1558.1, tag_id_1576.1, tag_id_1598.1, tag_id_1604.1, tag_id_1613.1, tag_id_162.1, tag_id_1652.1, tag_id_170.1, tag_id_1708.1, tag_id_1748.1, tag_id_1751.1, tag_id_1762.1, tag_id_179.1, tag_id_1804.1, tag_id_1808.1, tag_id_1810.1, tag_id_1836.1, tag_id_1850.1, tag_id_1880.1, tag_id_1889.1, tag_id_1915.1, tag_id_1950.1, tag_id_1961.1, tag_id_1966.1, tag_id_1982.1, tag_id_1994.1, tag_id_1999.1, tag_id_2008.1, tag_id_2009.1, tag_id_2017.1, tag_id_2062.1, tag_id_2082.1, tag_id_2114.1, tag_id_2134.1, tag_id_2155.1, tag_id_2178.1, tag_id_2182.1, tag_id_220.1, tag_id_2203.1, tag_id_221.1, tag_id_2214.1, tag_id_2237.1, tag_id_2247.1, tag_id_2258.1, tag_id_2301.1, tag_id_2319.1, tag_id_2368.1, tag_id_2499.1, tag_id_250.1, tag_id_2607.1, tag_id_2635.1, tag_id_265.1, tag_id_325.1, tag_id_402.1, tag_id_410.1, tag_id_436.1, tag_id_55.1, tag_id_572.1, tag_id_67.1, tag_id_770.1, tag_id_788.1, tag_id_843.1, tag_id_855.1, tag_id_874.1, tag_id_875.1, tag_id_879.1, tag_id_913.1, tag_id_942.1, tag_id_981.1, tag_id_987.1

54 Reporting Units: aleutianus, alutus, auriculatus, aurora, babcocki, borealis, caurinus, chlorostictus, constellatus, crameri, dallii, diaconus, diploproa, elongatus, emphaeus, ensifer, entomelas, flavidus, hopkinsi, jordani, maliger, melanops, melanostictus, melanostomus, miniatus, moseri, mystinus, nebulosus, nigrocinctus, oculatus, paucispinis, pinniger, polyspinis, proriger, rastrelliger, rosaceus, ruberrimus, rubrivinctus, rufinanus, rufus, saxicola, semicinctus, serriceps, umbrosus, zacentrus, reedi, wilsoni, atrovirens, carnatus, chrysomelas, goodei, levis, ovalis, serranoides

54 Collections: aleutianus, alutus, auriculatus, aurora, babcocki, borealis, caurinus, chlorostictus, constellatus, crameri, dallii, diaconus, diploproa, elongatus, emphaeus, ensifer, entomelas, flavidus, hopkinsi, jordani, maliger, melanops, melanostictus, melanostomus, miniatus, moseri, mystinus, nebulosus, nigrocinctus, oculatus, paucispinis, pinniger, polyspinis, proriger, rastrelliger, rosaceus, ruberrimus, rubrivinctus, rufinanus, rufus, saxicola, semicinctus, serriceps, umbrosus, zacentrus, reedi, wilsoni, atrovirens, carnatus, chrysomelas, goodei, levis, ovalis, serranoides

8.55511199822577% of allelic data identified as missing

assignment accuracy?

assign4 %>%
  filter(scaled_likelihood > 0.95)

986/1002

[1] 0.9840319

98.4% assigned correctly at the 95% likelihood threshold.

“incorrectly” assigned.

assign4 %>%
  filter(scaled_likelihood > 0.5 & scaled_likelihood < 0.95) %>%
  ungroup()

Summary: In this dataset, there are 1002 samples and 54 species. They are all adults with the exception of 47 juvenile samples included for crameri, wilsoni, and reedi.

I removed a total of 16 samples because of z-score < -3, half of which were hopkinsi, followed by 2 carnatus and 1 each of flavidus, levis, melanops, nigrocinctus, ovalis, and umbrosus.

z_bummers %>%
  bind_rows(z_to_remove) %>%
  group_by(collection) %>%
  tally %>%
  arrange(desc(n))

down_no_z_data %>%
  left_join(., meta, by = c("indiv" = "NMFS_DNA_ID")) %>%
  select(indiv, collection, REPORTED_LIFE_STAGE) %>%
  filter(REPORTED_LIFE_STAGE == "JUVENILE") %>%
  unique()

Hopkinsi investigation

# closer look at hopkinsi
minus_juvs_selfass %>%
  filter(scaled_likelihood > 0.5) %>%
  #filter(z_score < -3) %>%
  #select(indiv, collection, inferred_collection, z_score, scaled_likelihood, n_miss_loci) %>%
  filter(collection == "hopkinsi") %>%
  left_join(., meta, by = c("indiv" = "NMFS_DNA_ID")) %>%
  select(indiv, REPORTED_LIFE_STAGE, COLLECTION_DATE, SAMPLE_COMMENTS)

What if I have sibling groups in my hopkinsi data? Are the “larvae” the hopkinsi samples with low z-scores?

minus_juvs_selfass %>%
  filter(scaled_likelihood > 0.5) %>%
  filter(z_score < -3) %>%
  #select(indiv, collection, inferred_collection, z_score, scaled_likelihood, n_miss_loci) %>%
  filter(collection == "hopkinsi") %>%
  left_join(., meta, by = c("indiv" = "NMFS_DNA_ID")) %>%
  select(indiv, z_score, COLLECTION_DATE, SAMPLE_COMMENTS)

Yes. Not totally sure what to make of this.

Make a list of those samples to investigate this elsewhere:

Here’s another thought:

Take a look at the meta data notes to see if the term “larvae” comes up more than just in the hopkinsi

try2 %>%
  left_join(., meta, by = c("indiv" = "NMFS_DNA_ID")) %>%
  #select(SAMPLE_COMMENTS, REPORTED_LIFE_STAGE, COLLECTION_DATE) %>%
  filter(!is.na(SAMPLE_COMMENTS)) %>%
  filter(SAMPLE_COMMENTS != "Already extracted DNA.") %>%
  filter(SAMPLE_COMMENTS != "Already extracted DNA plate.") 

NA

meta %>%
    filter(!is.na(SAMPLE_COMMENTS)) %>%
  left_join(., try2, by = c("NMFS_DNA_ID" = "indiv")) %>%
  filter(collection == "hopkinsi")

NA

It’s pretty ambiguous whether the samples are from adults (females) who had larvae, or if the samples were larvae themselves and shoud not be included.

For a moment, let’s assume that the samples that say “Larvae” are referring to a female who has larvae.

But if sample R010389 says “Genotype identical to R010388”, then I should make sure I do a matching samples analysis.

Hopkinsi for COLONY

Grab the hopkinsi data for COLONY input

new2col %>%
  filter(collection == "hopkinsi") %>%
  select(-sample_type, -repunit, -collection) %>%
  write_csv("csv_outputs/hopkinsi_genos.csv")

And read in the output (with the caveat that we probably don’t have the power to actually distinguish sibs - low hz)

Based on that, it looks like there is one pair of sibs and one group of four full-sibs.

How do those compare to the hopkinsi samples that had the z-scores?

hopk_sibs <- sibs %>%
  filter(Probability > 0.95) %>%
  filter(ClusterIndex %in% c(2, 3)) %>%
  left_join(., hopkinsi, by = c("OffspringID" = "indiv")) %>%
  select(ClusterIndex, OffspringID, z_score, SAMPLE_COMMENTS) %>%
  select(OffspringID) %>%
  rename(indiv = OffspringID)